[Docs]Document clarifying notes about the data lifecycle #5922
Conversation
Signed-off-by: Alex Wu <[email protected]>
Signed-off-by: Alex Wu <[email protected]>
Signed-off-by: Alex Wu <[email protected]>
| Flyte does not offer a direct function to delete raw data stored in external datastores like ``S3`` or ``GCS``. However, you can manage deletion by configuring a lifecycle policy within your datastore service. | ||
|
|
||
| If caching is enabled in your Flyte ``task``, ensure that the ``max-cache-age`` is set to be shorter than the lifecycle policy in your datastore to prevent potential data inconsistency issues. | ||
|
|
||
| Practical Example |
There was a problem hiding this comment.
Not sure how this example connects with the content immediately above. This is an example of data movement, which is good, so maybe it would be better placed at the beginning of the page?
There was a problem hiding this comment.
Hi @davidmirror-ops , I think you are right. The example section in this PR is indeed more relevant to data movement. To reduce ambiguity, I’ve moved the example section directly below the data movement section and relocated the Own Datastores section to the bottom of the page. Let me know if you have any further suggestions, thanks! doc-link
Signed-off-by: Alex Wu <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5922 +/- ##
==========================================
+ Coverage 36.53% 36.84% +0.31%
==========================================
Files 1249 1309 +60
Lines 128230 130994 +2764
==========================================
+ Hits 46847 48268 +1421
- Misses 77346 78536 +1190
- Partials 4037 4190 +153
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Tracking issue
Closes #4683
Why are the changes needed?
What changes were proposed in this pull request?
Check all the applicable boxes
Docs link
https://flyte--5922.org.readthedocs.build/en/5922/user_guide/concepts/main_concepts/data_management.html